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. . . We explore low temperature properties of quantum triangular Heisenberg antiferromagnets in 

' two dimension in the vicinity of the quantum phase transition at zero temperature. Using the 

' effective field theory described by the 0(3) x 0(2) matrix Ginzburg-Landau-Wilson model and the 

, non-perturbative renormalization group method, we clarify how quantum and thermal fluctuations 

(N . affect long-wavelength behaviors in the parameter region where the systems exhibit a fluctuation- 

$H ' driven first order transition to a long-range ordered state. We show that at finite temperatures the 

crossover from a quantum (f>^ theory to a renormalized two-dimensional classical nonlinear sigma 
model region appears, and in this crossover region, massless fiuctuation modes with linear dispersion 
a la spin waves govern low-energy physics. Our results are partly in good agreement with the recent 
experimental observations for the two-dimensional triangular Heisenberg spin system, NiGa2S4. 
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I. INTRODUCTION 



, It has been argued for decades that geometrical frustration gives rise to unusual magnetic properties in quan- 
tum antiferromagnets, and may bring about an exotic ground state such as a spin liquid which is character- 
ized by the absence of any type of spontaneous symmetry breaking including magnetic long-range order (LRO), 
^ ' and dimerization, etC]i*2i^*^iS*^ The concept of the spin liquid was first proposed by Anderson in connection 
\ with the ground state of two-dimensional (2D) Heisenberg antiferromagnets (HAF) on a triangular latticed 
P5 ■ While extensive studies on frustrated magnets have revealed that the triangular HAF may exhibit the magnetic 
O L]^Q^8j9ji0jiiji2ji3ji4ji5ji6ji7jif -j-j^g Andcrson's idea is still attracting much interest, and has been tested for its possible 
, realization in other geometrically frustrated systems such as pyrochlore and Kagome HAF, and multiple ring exchange 

spin ni3 del: . 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35 

^SJ , Recently, Shimizu et al. reported that in an organic Mott insulator with spin s = 1/2 on a triangular lattice, 
' K-(ET)2Cu2(CN)3, no magnetic LRO is observed down to 32 mK.'^S Their experimental results suggest the possibility 
^ : of a new kind of a ground state including the spin liquid. Subsequently, Nakatsuji et al. found that in a quasi-2D 
quantum triangular HAF with s = 1, NiGa2S4, there is no sign of LRO down to 0.35 K, in spite of the existence 
of strong anferromagnetic interactions.'^^ In the latter system, the specific heat coefficient shows the quadratic- 
temperature dependence Cy ~ at sufficiently low temperatures, and the uniform spin susceptibility is constant in 
' the low temperature regions, indicating the existence of low-lying massless excitations. The origin of these unexpected 
low-temperature behaviors has not yet been explained. 

On the other hand, from theoretical point of view, there have been only a few works on low-energy properties in the 
] vicinity of the quantum phase transition at T = in the 2D triangular HAFi22i2S To understand the above-mentioned 
' experimental observations for quantum triangular HAF precisely, we need to develop a theory which describes quantum 
critical phenomena in these systems at finite temperatures. In contrast to the quantum case, the three-dimensional 
(3D) classical stacked Heisenberg model, which is equivalent to the 2D quantum model at T = 0, has been extensively 
studied by many authorsi^i'^i^°i^^i^^i^'^i^^i^^i^^i^^i-^^i^^i^^i^^i^^i'^^i^Q Even for the classical systems, the elucidation of the 
Q : nature of the phase transition has not yet been completed. Several theoretical works done by Zumbach, Loison and 
ILJ ■ Schotte, Itakura, and Delamotte et al. indicate that the 3D classical triangular HAF show a fiuctuation-driven first 
. ^ \ order phase transition4i*^2ii2i^^4^ii& On the other hand, the loop expansion calculations carried out by Pelissetto et 
■ al. and Calabrese et al. support the existence of the continuous phase transition in these systemsi^^i^ Also, most of 
5h ' experiments seem to be in accordance with the continuous transition.^" However, the former point of view is quite 
. . I intriguing, since it implies that the phase transition of the 2D quantum version of these systems at T = may be the 
(quantum) fluctuation-driven first order type. Although the quantum second order phase transition and the related 
quantum critical phenomena have been comprehensively explored so far^^Si^SiiSiSiSiS long-wavelength properties 
which emerge near the quantum fluctuation-driven first order transition have not been clarified sufficiently. It is 
well-known that in the case of the continuous quantum phase transition, critical phenomena just above the transition 
point at T = are described by a renormalized 2D classical theory. In contrast, it is highly non-trivial how quantum 
and thermal fluctuations which induce a flrst order transition at zero temperature affect low-energy behaviors (See 
Fig-D- III this paper, we would like to address this issue for the 2D quantum triangular HAF. 

Generally, in 2D quantum critical phenomena, the crossover from the 2+lD quantum behaviors to the renormalized 
2D classical ones occurs at finite temperatures. For the precise description of the quantum-classical crossover, a 
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FIG. 1: Phase diagrams of quantum second order phase transition (top) and quantum first order transition (bottom). The 
vertical and horizontal axes are, respectively, temperature T and a parameter Xjn which controls quantum fluctuations. 



promising theoretical approach may be the non-perturbative renormalization group (RG) method. This technique 
has been applied to classical frustrated magnets by Delamotte et al. yielding fruitful results.'*^ They showed that the 
non-perturbative RG method successfully reproduces the RG equations for both the four-dimensional (4D) Ginzuburg- 
Landau- Wilson (GLW) model and the 2D nonlinear sigma model, which are the effective field theories for the 4D and 
2D classical triangular HAF, respectively, and is expected to capture correct low-energy physics of the 3D triangular 
HAF. Thus, the non-perturbative RG method may be suitable for the investigation of the dimensional (or quantum- 
to-classical) crossover phenomena for these systems. We utilize this remarkable merit of the approach to shed light 
on how quantum and thermal fluctuations control low-energy properties in the vicinity of quantum fluctuation-driven 
first order transitions. 

Our main results are as follows. At finite temperatures, the crossover from a quantum 0^ model to a renormalized 
2D classical system appears, and as T decreases, long- wavelength behaviors are almost governed by the t/)^- fluctuations 
which eventually bring about the first order transition. Also, it is found that in this crossover region, thermodynamic 
properties are effectively determined by gapless excitations a la spin waves with a linear dispersion. The presence of 
these low-lying excitations explains partly the experimental observations for NiGa2S4; i.e. the quadratic T-dependence 
of the specific heat coefficient, and the finite T-independent uniform spin susceptibility at low temperatures >^ 

The organization of this paper is as follows. In Sec. II, the effective field theory and the formulation of the non- 
perturbative RG method are given. In Sec. Ill, we present results on the RG flows obtained by solving numerically 
the RG equations, which demonstrate the existence of strong fluctuations driving the phase transition at T = to 
the first order type. In Sec. IV, we show that in the crossover region, quasi-Gaussian fluctuations dominate low-energy 
properties, and derive low-T behaviors of the specific heat coefficient and the uniform spin susceptibility. Summary of 
our results and discussion on the implication for the recent experimental observations for NiGa2S4 are given in Sec.V. 
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II. EFFECTIVE FIELD THEORY AND NON-PERTURBATIVE RENORMALIZATION GROUP 

METHOD 

A. Quantum 0(3) x 0(2) matrix Ginzburg-Landau- Wilson model for frustrated magnets 

Quantum phase transitions in two dimension which occur at T = can not be described by the usual 0^-type 
Ginzburg-Landau scheme, since the long-range ordered phase exists only at T = 0, and the order parameter becomes 
nonzero abruptly just at T = 0. In the case of 2D quantum non-frustrated HAF, the low-energy properties of the 
quantum phase transition are successfully explained in terms of the 0(3) nonlinear sigma modeli^ In the derivation of 
the nonlinear sigma model from the HAF, one merely postulates that the correlation length is larger than the lattice 
constant. Thus the model describes long- wavelength physics of both the ordered phase at T = and the disordered 
phase above the transition temperature, which are mainly governed by transverse fluctuations a la spin waves. The 
nonlinear sigma model is suitable for the description of the 2D quantum phase transition in which the order parameter 
amplitude is almost frozen, but the strong transverse fluctuations destroy the LRO at finite temperatures. 

The extension of the nonlinear sigma model approach to the 2D quantum triangular HAF was achieved by Dombre 
and Read, and Azaria et al^^^ The order parameter for this case is expressed by a matrix with 0(3) x 0(2) symmetry, 
$ = (01,02), where 0- = (0ii, 02i, 03i) is an 0(3) vector. The 0(3) symmetry reflects the spin rotational symmetry. 
The 0{2) symmetry comes from the E representation of the Csv symmetry of the triangular lattice, which is enlarged 
to 0(2) in the continuum limit4SiSi These vectors obey the non-linear conditions, |0ip = 1, (f>i ■ ip2 — corresponding 
to the 120° structure ordered state. 

Subsequently, however, it was recognized that the 0(3) x 0(2) non-linear sigma model in three dimension is not 
a proper low-energy theory for the classical stacked triangular HAF, because the model neglects totally longitudinal 
(amplitude) fluctuation modes, which importantly induce a fluctuation-driven flrst order transitioniSiii^ This 
fact implies that the 2D quantum version of the 0(3) x 0(2) nonlinear sigma model fails to capture the important 
low-energy physics at T = 0. To improve the model, one needs to introduce an effective potential which replaces the 
above nonlinear condition with the relaxed constraints. Then, the correct low-energy effective theory for 2D quantum 
triangular HAF is given by the 0(3) x 0(2) matrix Ginzburg-Landau- Wilson (GLW) model, of which the action is 
given by. 



S — Sa + 5*4 + Sq, 



(1) 




(2) 




(3) 




(4) 



with P — 1/T the inverse temperature, and 



14 = ey($'5a$) 



(5) 




Here, 9^ = {-^dr,dx,dy), da = {-^dr,dx,dy). The model describes two massless excitations with the velocity ci, 
and a massless excitation with the velocity. 




(7) 



as well as a massive excitation with the mass Ak, and two massive excitations with the mass flk The three massless 
excitations are, respectively, two out-of-plane modes and one in-plane mode of transverse fluctuations a la spin waves. 
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The second part 5*4 is an effective potential which imposes the released non-linear conditions on 4>i. In the limit of 
A ^ oo, p, ^ oo, S4 recovers the strict nonlinear conditions, and S'^- + S4 becomes equivalent to the action of the 
0(3) X 0(2) nonlinear sigma modeliiiiSSiSiiS The release of the constraint allows the existence of the three massive 
longitudinal fluctuation modes. The classical version of the model S'o.-l-5'4 has been extensively studied so fari^2iHiiSii& 
Here we also consider the 6-body part Sq which is required for the correct description of the fluctuation-driven first 
order transition. In the expression of 5*6, we neglect terms with derivatives, because the scaling dimensions of these 
terms imply that they are irrelevant. Although in the following our analysis is applied to non-perturbative regions 
including the strong coupling limit, we believe that the omission of the 6-body terms with derivatives would not change 
the essential features of our results. Then, since any polynomials of (pi which preserve the 0(3) x 0(2) symmetry are 
expressed in terms of p and 6-body terms are generally given by eq.Q). The effective field theory captures 
low-energy physics of the 2D quantum triangular HAF in the vicinity of the long-range ordered state with the 120° 
structure, which are governed by both quantum and thermal fluctuations. The 120° structure state is expressed by 
the conflguration of which minimizes S', i.e. SS/dipi = 0. Transverse and longitudinal spin fluctuations around this 
configuration which preserve relative angles between spins on a primitive triangle are included in the model . 

It is noted that although the condition SS/6(j)i = leads the finite amplitude of the order parameter, the model is 
also applicable to the disordered phase at finite temperatures, since in 2D systems transverse fluctuations for T > 
are so strong that the average magnetization vanishes even under this condition, in accordance with the Mermin- 
Wagner-Coleman theoremi^ This is a key feature of the nonlinear sigma model as an effective field theory for the 
2D quantum phase transition at T = 0. We would like to stress again that any perturbative approaches for the 
GLW model ^ can not describe the 2D quantum phase transition properly because of the reason explained above, 
and a theoretical framework which can interpolate the nonlinear sigma model and the GLW model is required. The 
non-perturbative RG method is most suitable for this purpose, as will be explained in the following sections. 



B. Non-perturbative renormalization group method for quantum phase transitions 

We apply the non-perturbative RG method to the model This approach was developed by Wetterich, Zum- 
bach, and Delamotte et al. for classical matrix GLW models in connection with frustrated magnets 
A remarkable merit of this method is that in contrast to perturbative RG calculations, it is applicable to the whole 
range of the coupling constants A and fi including the strong-coupling limit, X, fj, 00. In fact, Delamotte et al. 
showed that the non-perturbative RG method successfully reproduces the RG equations for both the 4D GLW model 
in the weak coupling limit and the 2D non-linear sigma model in the strong coupling limit. This feature is quite 
important in the investigation of the quantum phase transition, since our systems may exhibit the quantum-classical 
crossover at finite temperatures, which is equivalent to the dimensional crossover from the 2D classical region to the 
2-|-l(=3)D quantum-fiuctuation-dominated region. We exploit this fascinating advantage of the non-perturbative RG 
method in the following. 

It is straightforward to generalize the derivation of the RG equations for the classical system to the 2D quantum 
caseiS2i*Li'62'63 assume that the effective action rfc[0] for any values of a scaling parameter k has the same form as 
eq.(l) with the renormalized parameters; i.e. T[(p] = S. This means that the effective action is truncated up to the 
6-body term. To argue quantum critical behaviors at finite temperatures, we utilize a non-relativistic renormalization 
schemCf^iiSi in which the infrared cutoff is introduced in the momentum space as a scaling parameter k. The exact 
renormalization group equation for the quantum effective action [0] is given by, 

p,p' 

where t — — Ink, p = (iujn^q). tUn is the Matsubara frequency 27rnT, and q is the 2D momentum. Rk{q^) is the 
infrared cutoff function in the momentum space, dt acts only on Rk{q^). In the following, we use the theta cutoff 
function, 

Rk{q^) = Z{e -q^)Q{e-q^). (9) 



(2) 

is the inverse one-particle Green function defined by, 



S(l)aip)S(l)p(j)') 



(10) 



Here, the derivative is taken at the configuration which minimizes Ffc[0] = S. 
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To investigate the renormalization group flow of the effective action (Q, we introduce dimensionless renormahzed 
coupHngSr^ 

Zk^^"^ ^ cqX co/i 

-^=z^' ^=Z^' ^''^ 



Here cq is an initial value of ci . The non-perturbative RG equations for these couplings and the velocities ci and C3 
are given in the appendix. In the next sections, we present the results obtained from numerical solutions of the RG 
equations. 



III. RENORMALIZATION GROUP FLOWS FOR DIMENSIONLESS COUPLINGS AND THE PHASE 

DIAGRAM 

For the characterization of the quantum phase transition, it is convenient to introduce a dimensionless renormalized 
temperature T' = T/{cok) with cq an initial value of ci. At finite temperatures, in the scaling limit k 0, the 
renormahzed Matsubara frequency w„ = 2TmT' for 71 7^ becomes infinity, and thus does not contribute to low-energy 
properties. Then, in this limit, the system is in the class of the 2D classical model, in which quantum effects are 
entirely included in the renormalization of parameters. As will be seen below, this renormalized 2D classical behaviors 
appear only in the sufficiently long- wavelength scale fc <C fee where the critical value of the scaling parameter kc is 
proportional to T. 

We solve the RG equations ljAl|l - ljA9p numerically for some particular sets of initial values of parameters by using a 
Runge-Kutta-Verner method with high precision. We put k — 1 at the initial stage of the renormalization. Depending 
on the initial values of the parameters, there are two regions in the scaling limit at T = as indicated in Fig.^ i.e. 
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FIG. 3: Effective potentials 54 + Se as a function of yfp calculated for a particular set of scaling parameters fc under the 
condition f = 0. 



a long-range ordered phase, and a quantum disordered phase (quantum paramagnet). The value of 1/k at the phase 
boundary depends on the choice of the initial values of the other parameters. The correspondence between the spin S 
triangular HAF with the nearest-neighbor exchange interaction J and the 0(3) x 0(2) non-linear sigma model implies 
that the initial values of parameters are set 

Cl — — ■=,Jba, K — Jb , Z = 1, 

2V2 4 ' 



Cj 16 
c? 27 VS.PS^a'^' 



'^ = 0' 3 = - .^ o ' (13) 



where a is the lattice constant. These parameters are in the region where the LRO exists at T = 0, and thus we 
concentrate on this case in the following. As will be seen below, as long as the initial values of parameters are in this 
region, the qualitative and essential features of the RG flows are not altered by changing the parameters from those 
given by ((T^ . 

In Fig. 12 we show the RG flows obtained at T = O.OOIcq for some initial values of parameters. Since the parameter 
At, which is the overall coefficient of the action (Q, scales to infinity as fc ^ 0, we plot the running coupling constants 
of the 4-body and 6-body terms, Ak^, /zk^, Xqk^, and /Xe^'^, divided by k to specify the low-energy behaviors. The 
numerical solutions for the RG equations (|A1|) - (|A9|) show that the nature of the phase transition and long- wavelength 
behaviors of the model are different from those predicted for the nonlinear sigma modelj^ We do not find any non- 
trivial fixed points which separate the long-range ordered and disordered states, in agreement with the recent studies 
on the 3D classical stacked triangular iiAF42ii2ii4iiS4& As the scaling parameter k decreases, Xk and iik scale to 
large values, implying that at finite temperatures the system is renormalized toward the class of the renormalized 2D 
classical nonlinear sigma model. However, it should be noted that in the intermediate region, the 6-body fluctuations 
characterized by the parameters AgK^ and hqk^ develop strongly, which may eventually induce a fiuctuation-driven 
first order transition at T = 0. 

To see the crossover behavior toward the (j)^ model in the intermediate scale more clearly, we depict the renormal- 
ization of the effective potential 5*4 -|- Sq for some values of the scaling parameter A; at T = O.OOIcq in Fig. |31 Here, 
for simplicity, we plot the effective potential as a function of y/p under the condition f = 0. In the early stage of 
the renormalization (k=0.818 in FIG. 3), the effective potential has two minima at y/p = ±-\/2, corresponding to the 
nonlinear condition which expresses the situation that longitudinal fluctuations are suppressed, but strong tranverse 
fluctuations inhibit the emergence of the LRO. As k decrease, a minimum at -^p — appears, and the depth of the 
valley at the origin becomes deeper and deeper, indicating that the fluctuations which may drive the phase transition 
into the first order type is developing. At fc = 0.00673 in FIG. 3, however, the growth of the minimum at the origin 
stops and turns to a decline for k < 0.00673. Eventually, for sufficiently small k {k = 0.00012 in FIG. 3), the minimum 
at the origin disappears and the potential valleys at finite ^/p become deeper. It is noted that at this final stage the 
effective potential does not describe a (/)* theory, but corresponds to the nonlinear condition, showing that the system 
is scaled to the renormalized 2D classical nonlinear sigma model. Here we introduce the scale kc which separates the 
region in which ^^-type fluctuations strongly develop (fc > kc) and the renormalized 2D classical region(fc <C kc). It 
is convenient to define kc as the value of k for which the potential depth at the origin and that at a finite value of 
^/p coincide. (In Fig. |3I = 0.00024.) At fc = kc, the paramagnetic state and the magnetically ordered state are 
degenerate. If this situation realizes in the limit of fc ^ 0, the first order transition to the ordered state occurs. Indeed, 
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FIG. 4: Plot of the critical scaling parameter fcc versus temperature T. 
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FIG. 5: A schematic phase diagram. The inserted plots are the renormalized potential versus the renormalized field y/p. 



this happens at T = 0. In Fig. 01 we plot kc calculated for several values of temperatures T . The results show that 
kc is proportional to T, and the true phase transition which occurs in the limit fc — > realizes only at T = as a first 
order transition, which is consistent with the results derived for the 3D stacked classical model42ii4iSii£ fc^, ^ T is also 
the scale at which the quantum-classical crossover occursiS Then, as the temperature decreases toward T = 0, the 
renormalized 2D classical behaviors appear only for length scales much larger than 1/fcc, and the low-energy physics 
at finite temperatures are mainly governed by the quantum fluctuations which induce the first order transition. In 
Fig. O we show the schematic phase diagram on the T-1/k plane suggested from these RG flows for a particular set 
of the initial values of the other parameters, A, /x, etc. The essential feature of the phase diagram depicted in Fig. [S] 
does not depend on the choice of these initial values. It is noted that in the long range ordered state at T = 0, the 
effective potential has degenerate minima at the origin and at the non-zero magnetization. This is the unique feature 
of the quantum first order phase transition for which the transition temperature is T = 0. The shape of the effective 
potential implies that there may be a possible coexistence of the ordered and disordered states. The fluctuations 
which lead the crossover toward the (j)^ theory should significantly affect the low-energy properties of the system. We 
would like to address this issue in the next section. 
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IV. EMERGENT QUASI-GAUSSIAN LOW-ENERGY BEHAVIORS 

Since the zero temperature phase transition in our system is the first order type, universal critical behaviors do not 
exist. However, it is still possible to relate low-energy properties of the model ^ appeared in a certain parameter 
region to experimentally observable quantities which are governed by quantum and thermal fluctuations. For this 
purpose, we investigate the RG flows of dimensionful couplings k, A, /i, a), Ae, and p,e for some particular sets of 
initial values of the parameters, from which physical quantities can be calculated. Surprisingly, as we will see below, 
long- wavelength qualitative behaviors of these running couplings seem to be almost universal to some extent at least 
in the region where the long-range order realizes at T = 0, and furthermore, the effective action Tklcj)] is renormalized 
to a system in which Gaussian fluctuations dominate low-energy properties. 

In Figs. El and we show the RG flows for the dimensionful couplings. It is noted that A, fl, Ae, and /ig scale to 
zero, and k and uj scale to finite non-universal constants. Besides, k 3> k'^u) for fc ^ 0, as long as the initial value 
of k'^dj is much smaller than that of k, which is a proper assumption for our system since the w-term (the second 
term of eq.©) is generated in the process of the renormalization. We would like to stress that these characteristic 
behaviors are rather universally found for any initial values of the parameters in the region mentioned above. These 
observations imply that in the scaling limit A: ^ 0, the effective action is renormalized to a Gaussian-like model, which 
is given by the first term of eq.||2Jl without the nonlinear conditions: 

2 

- / d^x J dT^ J2 ^M'?^ ■ (14) 

In the region where the Gaussian-like fluctuation, i.e. the free boson with a linear dispersion, dominates, the specific 
heat coefficient is easily calculated as, 

a = (3V3/7r)C(3)(T/ci)2. (15) 

The spin susceptibility is also obtained from the Gaussian action. For this purpose, we introduce an external in-plane 
magnetic field h, which couples with the uniform component of spin fluctuations in the form, 

- ■ {(fl X dt$i +$2^ dt$2)- (16) 
^1 

Then, in the quasi-Gaussian region, the spin susceptibility in the limit T — )■ is a non-zero constant given by, 
X = —d^F/dh^ — A:o/(127rci) with fcg an ultra-violet momentum cutoff. The quasi-Gaussian modes behave like spin 
waves with a linear dispersion, despite the absence of spontaneous symmetry breaking at finite temperatures. These 
results seem to explain partly the recent experimental observations for the quasi-2D triangular HAF NiGa2S4 at low 
temperatures. The spin-liquid-like behaviors found in this material at finite temperatures may be attributed to the 
existence of these fluctuation modes. 



V. SUMMARY AND DISCUSSION 

We have investigated low-energy properties of the 2D quantum triangular HAF using the non-perturbative RG 
method and the mapping to the 0(3) x 0(2) matrix GLW model. Our findings are as follows: 

(i) At finite temperatures, at the length scale shorter than 1/fcc the 0^ model which describes the fluctuations 
driving the transition to the first order type dominates the low-temperature behaviors, while at the scale larger than 
1/fcc the renormalized 2D classical region appears. As the temperature is lowered, the crossover scaling parameter 
decreases as (x T, and eventually at T = 0, the first order phase transition to the 120° structure occurs. 

(ii) In the crossover region at finite temperatures, the long- wavelength properties are governed by quasi-Gaussian 
fluctuations a la spin waves with a linear dispersion, which give a quadratic T-dependence of the specific heat 
coefficient and a finite and T-independent non-zero value of the uniform spin susceptibility at low temperatures. 

Here we would like to discuss the relation between our results and the recent experimental observations for the 
quasi-2D quantum Heisenberg antiferromagnet on a triangular lattice, NiGa2S4, which show no sign of LRO down to 
37 mK, implying the possible realization of a spin liquid. At low temperatures, the system exhibits some remarkable 
properties; (a) The specific heat coefficient shows the quadratic temperature dependence oc T^, indicating the 
existence of gapless excitation modes, (b) The uniform spin susceptibility for T is a nonzero constant, suggesting 
that magnetic excitations are gapless. (c) In contrast to the observations (a) and (b), the magnetic correlation 
length measured by the neutron scattering is rather short; i.e. ^ ~ 2.5 nm, leading the authors of ref. 37 to the 
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FIG. 6: RG flows of dimensionful couplings for some particular sets of the initial values of parameters, (a) Plot of k, X, and 
fl. (b) Plot of a), Ae, and fie- The initial values of parameters are, k{t = 0) = 0.3, A(0) — 0.5, p.{0) — 0.1, ijj{0) = —0.0001, 
a)(0)/ci(0) = -0.5, ci(0) = 1.0, (solid line); k(0) = 0.3, A(0) = 0.3, /i(0) = 0.22, ii(0) = -0.0001, c;3(0)/ci(0) = -0.5, 
ci(0) = 1.0, (broken line); ft(0) = 0.3, A(0) = 0.3, /i(0) = 0.3, Q{0) = -0.0001, £i(0)/ci(0) = -0.5, ci(0) = 1.0, (thin broken 
line); k{0) = 0.3, A(0) = 0.3, fi{0) = 0.4, u{0) = -0.0001, ii(0)/c3(0) = -0.5, ci(0) = 1.0, (dotted line); k{0) = 0.3, A(0) = 0.3, 
/i(0) = 0.5, £l;(0) — —0.0001, iIj(0)/c|(0) = —0.5, ci(0) = 1.0, (dotted-and-broken line), and the initial values of both Ae and fxe 
are equal to in all calculations. 

conclusion that there may exist gapless non-magnetic modes. Our results for the specific heat coefficient and the 
spin susceptibility obtained in Sec. IV seem to be in agreement with the observations (a) and (b), provided that the 
origin of the gapless excitations may be attributed to magnetic ones, and that in this system the coupling between 
the triangular layers is so week that thermal fluctuations suppress the true LRO at experimentally accessible low 
temperatures. However, the observation (c) implies that the magnetic excitations may not propagate coherently and 
have an excitation gap, and is not consistent with our RG analysis which shows the existence of an exponentially long 
correlation length. Then, how to reconcile our results with the observation (c)? To explain this point, we would like to 
note that in our field-theoretical model, chirality domains, which inevitably exist in real triangular HAF and suppress 
the development of the magnetic correlation length, are not included. Also, as was pointed out by Kawamura and 
Miyashita^, Z2 vortex, which is not taken into account explicitly in the field-theoretical model, may play a crucial 
role at finite temperatures, disturbing the growth of ^.^^ It is expected that the velocities of the magnetic excitations, 
ci^3, are strongly renormalized by these topological defects, and reduced to small values. Then, as long as ^ > c/T, 
the magnetic excitations behave like gapless, and our results may be applicable. Indeed, the experimental observation 

(b) intimates the existence of gapless magnetic excitations. Also, to interpret the experimental observations (b) and 

(c) in a consistent way, one might need to consider effects of randomness such as impurities, which are beyond the 
scope of this paper, but may play a crucial role in connection with the topological defects inherent in triangular HAF. 
To confirm this speculation, we need further studies. We would like to address this issue in the near future. 
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FIG. 7: RG flows of dimensionful couplings. The initial values of parameters are, k{t = 0) = 0.3, A(0) — 3.0, /i(0) — 3.0, 
u){0) = -0.0001, (l;(0)/ci(0) = -0.5, ci(0) = 1.0, (solid line); k{0) = 0.3, A(0) = 5.0, fl{0) = 5.0, ^{0) = -0.0001, ti(0)/ci(0) = 
-0.5, ci(0) = 1.0, (broken line); S(0) = 0.3, A(0) = 8.0, /i(0) = 8.0, (I)(0) = -0.0001, £i(0)/ci(0) = -0.5, ci(0) = 1.0, (thin 
broken line); k{0) = 0.3, A(0) = 10.0, /i(0) = 10.0, ti(0) = -0.0001, u{0)/4{0) = -0.5, ci(0) = 1.0, (dotted line); k{0) = 0.3, 
A(0) = 12.0, /I(0) = 12.0, a;(0) = -0.0001, a>(0)/ci(0) = -0.5, ci(0) = 1.0, (dotted-and-broken line), and the initial values of 
both Ae and fig axe equal to in all calculations. 
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APPENDIX A: RENORMALIZATION GROUP EQUATIONS FOR THE QUANTUM 0(3) x 0(2) 

GINZBURG-LANDAU- WILSON MODEL 

In this appendix, we present the RG equations for the dimensionless renormalized couplings (|11|) and (|12|l for the 
effective action ^ as functions of the scaling parameter t — — Infc. Here we consider the general case of the spatial 
dimension d. Differentiating eq.® with respect to as was done in ref. 46, we end up with the RG equations 
truncated up to the 6-body terms, 

dtK = -{d-l + r,)n - ^h{n\) - h{n^J) - - (TV _ 2)^1(0) - - 2^h{K^i) 

- — [X6li{K\) + 2neli{Kfi)], (Al) 



dtX = ((i-3-H2ry)A- — (9Z2(KA) + 2Z2(KM)+r20 + 2(7V-2)/2(0)) 
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-2\ij,l2{Kfi) - Xu)r2i - a;^r22 - 2n^h{Kii) 

A« 

+4A6Zi(kA) + 68m6^i(km) - 4-f [wm + 2/iZi(K/i)] 

A 

-l%'^[\lh{KX) + 2XeH6li{KiJ.)] - 16XIk^12{kX) - 32ij,Ik^12{kh) 
A 

- 12kAA6/2 (kA) - 8k(A + 2n)nel2 {k/j.) , (A2) 

dtH = (d — 3 + 277)/! — 3/iAuii(KA, K/u) — /ua;riii(K/i) 

- ^[3uii(kA, K/i) + (AT - 2)Z2(0) - rno(K/x)] 

+ 4^6[^i('«A) + /i(k^) - ^rii - ^Ii{k^)] - ^^[Ae/xe + 2/X6'i(k/x)] 

— 32k,^ ij,qUii{kX, Kn) — 8K,iie{X + 2ij)uii{kX, K/j,), (^3) 

~h{0) - ^/i2oM) + Ka;2[2/ini(0) + /^llo(0) + (I + Kw)/i222(0) + 2r^^i{0)] 

-kijl^Ii2o{kh) (A4) 



dtc^ = — 



1- ^ 

•^3. 



Zi(0) 



ci kA^ 

+ [-3I22o(kA) + 2;2io(kA) + 4;3oo(kA) - 4;3io(kA)] 

2 

+ -'^^ [-3/220 (km) + 212\q{k\x) + 4Z3oo(k/u) - 4/3io(k;a*)] 

2 2 

+ ^^[4%?-iii(0) -ri2i(0) +4ri22(0) - 4ri22(0) - 4ri32(0) + 4«;i32(0) 

- 4/ini(0) - 2/iin(0) - 2(1 + Ku;)/i222(0) - 2riii(0)], (A5) 

a; A^ 

dtUJ = {d-l + 2l])uj+—[li{KX)-li{0)]^—[li2Q(KX)+2{l+KOj)h22o{KX)] 

+ u;2[2/iin(0) + /iiio(O) + (1 + Ka;)/i222(0) + r^^iiO) 

g 

- 6/ii2i(KA) - (1 + K;a;)/i222(«A) - riii(/^A) - ^^^^^(/^A) 

8 8 

(a*«) + ^J^(^ - 2)/io3o(0)] - m^/i2o(«m) 

+ Aa;[6/ii2o(KA) + (1 + Ka;)/i22i(«A) + riio(«;A)], (A6) 



^T--[^120(kA) + 2(1 + KUj)h22o{KX) - % 



dtcs = ^^^^[/i2o(kA) + 2(1 + Kui)h22o{KX) - ^{-/i2o(kA) - 3/22o(«;A) + 2/2io(«A) 



+ 4/3oo(kA) - 4?3io(kA) - 3w22o(kA) - 10wi3o(kA) 

+ 4(1 + k;A)u;23o(k;A) + 12(1 + k;A)u;i4o(k;A)}] 

+ ^[2/iin(0) + /iiio(O) + (1 + Kuj)h222{0) + riii(O) 

g 

- 6/ii2i(kA) - (1 + Kuj)h222{i^X) - riii(KA) - ^rjii(KA) 
+ rf^'^03o(M«) + ^(^ - 2)/io3o(0) 

- ^|{4|riii(0) - ri2i(0) + 4ri25(0)-4ri22(0)-4ri32(0) 

+ 4w;i32(0) + 6riii(KA) + 8wi2i(kA) - lQr^-^i{KX) - 16(1 + KA)ri2i(KA) 
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+ 16/io3i(M/«) + 16(7V-2)/io3i(0)}] 

2 2 

+ ^HMkX) - ^(-IMkX) - 3122o{kX) 
+ 2Z2io(kA) + 4Z3oo(kA) - 4Z3io(kA))] 

- ^ [-6/1120 (AtA) - (1 + «;a;)/i22i('«A) - rllO(KA) 

|{-3ri2i(KA) + 4ri2i(KA) - 4u;i3i(kA) + 4ri3i(KA)}], (A7) 

dtXe = {2d-4 + 3ri)Xe + 16K^Xll3{KX) -30kX112{kX) -UnneXehiKiJ,) 

- ZjiXehiKii) - ^^^r2i + 32«;^/x|Z3(k;/x) + ^hii^lA - "iSKulhiKfi) 

39/i6 

+ ^(2(^ - 2)/3(0) + 27Z3(kA) + 2k{Ki^) + no) 

27X^X6, , S/ieA^ , , 3mA^, , , 3wA2 
+ -^h{^X) + -^h{Kpi) + -^^h{Kix) + ^r3-, 

+ 18kAA^/3(kA) - ^(6(^ - 2);2(0) + 45i2(KA) + %h{n\i) + 3r2o) 
3Au^ 

+ 12K;A/ieZ3(K/i) H — - — h{Kii) - 6A/i6^2(KA*) 
4k 

, «\ ; / ^ , 3Aw^r32 ^^33 

+ 6A/^6M<3(km) H 5 h— — , (A8) 

8k 4k 

A^ 

dtiJ-6 = (2d - 4 + 377)/Lt6 + — (3u2i('«A, K^) + w2i(k/z, kA)) 

lb 

1, Uii(kA, K/Lt) 3?2(kA) hi^^i) 

8^ K ^ 2^ 2^ 

+ 4ku2i(kA, Kn)Xe + 24ku2i(kA, KiJ,)^e + 12ku2i(kM) i^X)fiQ 

+ 6U2l{KX,KfJ,)fI + 3U2l{KfJ,,KX)fJ,)X^ 

+ ^(96u2i(kA, K/i)/igK^ + 96'U2i('«/(x, KA)/igK^ + 64u2i(kA, Kfi.)XeiJ-eK^ 
8 

+ 16'U2i(kA, Kfj,)XefJ,K + 48u2i(kA, K^)^e^K 

+ 4:8u2i{kh, KX)ii(iiiK + 6'U2i(kA, kii)ii^ + 6m2i(k/x, kX)ij!^ + Z3(0)/U^ 
+ 4Z2(kA)A6 " 4/2(0)^^6 - 30/2(kA)^6 - 24l2{Kn)fie 

„ , , , , „ „ 2 / \ 2uii(kX,ku)u 

- Suii(kX, k^)(A6 + 4^6) - 2/i6y20 + 2a; r2i2[w H 

K 

_ 3/2(kA)/X _ /20l(«j»V _ /^?'20 _^ ^?^21 ^^^ 

2/^ 2/^ 2/^ 2/^ 

+ i(256,U6("2i(KA, K^)A6 + U2i{Kfi, kX)^q)k-^ 

+ 192u2i('«M, kX)ij,qHK^ + 128u2i(kA, KiJ,)XQiieHK^ 

+ 16u2i(kA, Kjj)XQlJL^ K, + 48U2i(k/Z, kX)iIqII^K — 16/L16(4mii(kA, Kjj)XQ 

+ 5/2(kA)A6 + 16uii(kA, K/U)/X6 + 10/2(K/U)M6)k 

+ 4u2i(k^, kA)/^^ - 4/2(kA)A6/X - 8l2(0)/U6At - S2l2{KIJ,)lJ,6fI 

- 16Wii(kA, K/x)(A6 + 4/i6)Ai - 41201 ('«m)M6'^ - 4/i6<^''21 

+ 4a;V2i3(«M) - + 



K K 

_^ 4?i(k/i)A6 _ 4/2(kA)A6 _ 44/i(k/^)/I6 _ ^20i(«m)m^ _ M^?'iio(km) 
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Ki Ki hh Ki 

Here, 77 — —dt In Z is the anomalous dimension, and the threshold functions appeared in the above expressions are, 



= ^T' ^ — ^ (AlO) 



2\n 



dyy'^-^T'Y — , (All) 

, (4 + l+C^^(2^+y2))™+l 



Vd^.^ -2[(m + n)(£^ + l) + mr + nX] 



= iT^'Y.- ? ' (A12) 



,(X)=vJ dyy'^-^rY,— j^^^ , (A13) 



^-3 



1 + X) 



(AM) 



ni -2[(m + n)(£^ + l) + mX + nwK(^+y2)] 
,(X) = z;, / dyy'^-'T'Y,— 2^ (A15) 



T (i^ + i + c^«(ii + y2))™+i(^ + i + x)"+i 

with Vd = d7r'^/2/[(27r)''r(d/2 + 1)], e„ = 27mr', and S = 4 + 2/^- r„i„s(X) is given by 1111511 with S replaced by y^. 

____ "^3 

rmns(-'^) is given by (|A15|I with S replaced by ef/c^- Also, Zm(X) = ImooiX). The renormalized temperature T' obeys 
the RG equation dtT' = —T'. In the absence of the 6-body terms, and in the limit T ^ 0, the above RG equations 
agree with those obtained by Delamotte et al. for the classical model truncated up to the forth order terms. The 
RG equations are non-perturbative in the sense that the beta functions (the right-hand sides of eas. ljAip - ljAQp 'l do 
not explode even in the strong coupling limit A — > 00, — )■ 00. The one- loop results of the nonlinear sigma model are 
reproduced in this limit. 
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